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ABSTRACT . . * ' 

7 Five questionnstire f«Srins cont/iining 61 items 
pp^ifyinq potential inservice topics for public school teachers were 
sent tc a stratified randcit Sample of Tnd-iana public school 
administrators and curriculum supervisors. The 'five forms differed in 
that, f.or t«c forms, the Items were unqrouped and appeared in 
different orders: >and, 'for three 'fcr.ms, the items appeiredsLn 
different orders and were presetited within diffejeat labeled^item 
qroupinos, .A 68 percent return rate provided 1 ,"U6 6 (iseable " - 
questionnaires^ for the data analysis. An item-order effect was 
demonstrated fcy comparinq Pearsot^ product-moment correlations," 
covariance matrices, residuals resulting from a fixed factor 
^structure, and factor leading patterns for the various questionnaire 
forms. The item-order effect appeared to be, more pronounced ^ for 
adiacent iteip pairs than for item- pairs separated by "one or wo 
items, Onlv a marginal effect was demonstrated for item grouping with 
captions. It fs probable that low-inference type items factor 
structures obtained from survey da^a are dependent upon item order, 
but only minimally affected by item grouping. (Author) 
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\ ABSTRACT 



. , Five questionnaire fomar containing 61 items speciiying potential 
Inservlce topics for public school teachers were sent to a stktlfied random 
«anple of Indiana Public School administrators and curriculum supervisors. 
Uie nye Ibnns differed In that, f9rvtWD forros, the items were -ungrouped 
^ appeared in different 6rder3; and, .for three forms, the item^ appeared ' 
.In dlfl^erent.-Crders and were p^sented within' different labeled item grcupings. 
- ^ . A 68 percent return rate pro|lde4 l,il68 usJable questionnaires, fbr the (Jat 
analysis^; An llten^^rder effebt was demDnsti^tec^ by conparing Pearson product r- 
moniant correlatUops, ^ovailance matrices, residuals resulting ftom a fixed 
ttQtol-- structure, and factor loading patterns for the variou^ questlorjnaire- 
Ibijns. Ihe Item-ord^r effect appeared ta be more pronounced for ,a>sljacent • 
Item pairs than for i^em pairs separated tor. one or two items. j3nly a 
nair^^lnal effect .was demonstrated for item grouping vdth captions.. 

it-Is piDbable that for' lqi««-lnferense type it^ factor structures 
obtained frcm survey data are, to- some extent dependent upon item order, "but 
oiily minimally affec.ted by item" grouping. " ♦ . * 
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Interest In the possible effects of Item order (seq^uence) ' 
on the analyses of questionnaire data stemmed from an examination of the 
, responses of a randoj^ sample of tedchers to a 61 Item Instrument designed 
to assess teacher perception of Inservlc^ training needs ^K. T. Schurr» 
F. Sclara» -et; al» 1978).. In attempting to Interpret a factor analysis of 
these data, the question arose as to whether the obtained Inter- Item corre- 
latl<^ns» and subsequent, factpr, analyses could be Influenced by the 

« 

sequancing and/or grouping of the Items. Of particular Interest was the 
possfble Influence of Item sequente on correlations qf adjacent pairs of 
Items. • . • . 

The particular results which piqued th'ls question were (l) 'tjie restdual 
correlations of adjacent Item palrs» after the extraction of factors, were 
fener^ally positive and larger than other residual correlations and (2) as 
might be suspected from this pattern, as the number of factors retained 
for rotation was Increased, Ihe ro'ta.ted factors consisted mostly of sets 

* s 

of f terns which appeared In sequence 6n the questionnaire. This pattern 
was fairly consistent regardless of the nurfiber lof factors extractj^aT That 
ts, using an addftlonaJ^ factor tended to spilt one sequjence of Items Into^ 
two sequencfa of Items, 

Since ail attempt had been made to prqvlde some logical organlzat/br#for 
the ttems, the factors with a rel'atlVely small number of Items cquIj^ b# g Iven 
a meaningful substantive Interpretation. However, the dilemma wa^w/^thei; a . 
solution consisting bf factors with a small number of Items bad iubfstantlve 
.meaning or whether these weaker factors were In fact an artl^^A^f^the man-* 
nar fn wh?^ the qi^estlonnalre was constructed** 
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A survey of the related literature Indicated a concern for ftem 
sequence or ordei* effects has been expressed by others (see Perrault, 
^976), but relatively few studies have been cjnducted" to assess the 
effects. Most of the expressed concern has been 'confined to the study 
of rtem order'as related to ttcms appearing early or toward the end 
of a a{ie?-tronnalre, where factors such as fatigue may become Important, 
or/the posiltlonlng ^ one Item relative tp another closely related Item, 

/ . . . ^ * . ... ... • . . . _ . ' .. .. 

.biasing responses to the latter Item. 

Sudman and Bradburn (1974) concluded that no theory regarding position 
effects could be formulated wl thout considerable additional research. Later, 
Bradburn and Sudman (1979) stated that litem yrder was of minor Importance In 
trj'luenclng responses obtained through the Interview technique. 

Most of ffieSftudles of Item order effects have been concerned with . 
either the comparison of mean Item response and/or proportions of respondents 
"agreeing" with a statement. Examples of Investigations for order effects 
ustng this type of analysis are provided by Krant, Wo If son, and Rothenberg 
0 975) and Clancy and Vachslcr (1971) . ^Krant et al. compared responses 
to kd Items placed near the beginning* of the questionnaire with responses 
to these Items when placed near the end of the questionnaire. They found 
that for Items measuring attitudes toward pay. Job security and advancement, 
respondents tended to choose extreme responses less frequently when an Item 
appeared near^the end of the-o^estlonna Ire and saw ttjis as an Important 
tonslderatlon for researchers comparing Information from one study to another. 
Clancy and Wachsler'' Investigated Item position by Inserting six agree-dlsagree 
type Items Into two versions of a shared-cost questionnaire (Items appeared 
near the beginning- In <Jne and near tfic end In the second version). They - 



3 
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concluded the magnitude of the effect was so small as to be of -no concern for 
that type of data. 

Metzner and Mann C1953) considered the problem of observed correla- 
tions for Items of a questionnaire being artificially Inflated as. a results 
of !tem grouping by using a sequenced (Items grouped and captioned by sub- 
Ject)/fnd nonsequenced (Items interspersed with other subject areas) form 
of a questionnaire. They were unable to conclude tha^ grouping with captions 
Ud to an Intensification of a correlation between Items. Kane (197T) , 
studied adjective scale order In the use of a s'emantfc differential. Using 
factor analytJc techniques and a comparison of factor loading patterns he 
concluded there were no significant Item order effects on the factor structure^ 

V P'^^ser^ study differs from that of Kane since he, used what Popham 

CI978, 196) terms b^gh- Inference Items and the questlonnlare employed 
In thts study contained low-Inference Items. It differs from the Metzner 
and Mann study In t^at a^larger number of Items was used and five forms 
rather than two were administered. The use of five forms permitted better 
estimates of the relative effects of Item grouping and sequencing, since* 
these Dwo effects were confounded In the correlations reported by, Metzner 



and Mann. 
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Method ' . * ^ 

Sample and Instrumentation * 

l>.t. for th, Investtgatlon were obtained as part of an ongoing stidy In 
which responses to the same 61 Items used In the study of teacher percept'lons 
wV. obtained from a stratified random.sample of Indlina Publ Ic'school admin- 
istrators, and curriculum supervisors. The purpose of this phase of the on- 
9or„9/itudy.w., to compa're teacher and admlnlstrator-superviwfjpirceptlons 
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of ?n$ervlce n*«ds. To Investigate the Item sequencing' and/or grouping ef- ^ 
fectCs), five distinct forms were constructed. Each form reflected a different 

'* ♦ * 

organ?zat?on of the 61 Items. * 

The 6f Items conslstedof brief descriptions of skills or activities that 
might serve as topics for Inserv^e training. The respondent was requested to 
rate each Item using the following scale: (1)- among the least useful; (2) Of 
Uttlc user (3) of moderate, use; {k) of_use., but not one of the most useful; 
and (5) among the Drost useful^. The Items are shown In Table 10. The five - , 
forms of the questionnaire consisted of the Items appearing In: (1) the same ^ 
seven Item groupings and In the|^ame sequence that was used In the teacher 
Study; (2) the same ^sequence tha.t was used In the teacher study, but un- 
grouped; (3) a random sequence; (A) seven groupings representing a plausible 
seven factor structure determined from the analysis -of , the teacher d^ta; and 
(5) nine of ten Item, groupings representing an alterna'tlve plausible factor 
structure determined from the analy?!? of the teacher data. • ' 

* * 

Five proportional random saTnjples of respondents, stratified accor<ilnq 
to their administrative-supervisory position, w^re selected. I^ch sample 
responded to a different form of the questionnaire.^ Name's of respondents 
wer.e obtained from an Indiana Department of Indiana tap^of all administrative 
•nd supervisory public school positions. The sample size for Form 1 
was two times as large as for the other forms. This was done to.as.surq that. 
In the d^ent that the al liernatjve forms affected administrator responses, a 
sufftctent number of responses to Form Kwould be obtained for a comparison 
to the teacher data. A 68 percent return rate, -Including one follow-up, 
resulted In 1,618 responses, of whlfch 1,^»68 were sufficiently complete to 

fnclude In the data analyses. ''Numb'fers of usable questionnaires by form were: 

■r , . ' -> • • • . • 

(1)Jf80,'C2) 2i|1. '6) 25i», 'CJ^r257. and <5) 236. 



Statistical Procedures * • / • \ 

■ - . . ■ , . • ^ ■ . - 

Four type* of procedures .were used ?n analyzlng*the obtained data: 
corpparing simple coi^ralat Ions, comparl^ng covarTance matrices, using conf Irma- 
• tory factor analysis procbdures 'and'^slng e-xploratory factor analysis pro-, 
cedures. The first procedure was to compute 'Pear son product -moment correla- 

y tlons for Item pairs that ;apHreafed' as adjacent" Item's on one form of the 

questionnaire, but nonadJacentM tpms on another form.. In particular corre- 
Utions Qorrespondlng to pairs of I terns adjacent on Form 3 (random Item ^ 
sequence) but pot on Form 2 were cqmpute<i and compared uslngi the procedure 
described In Glass and Stanley (1970, p. 32I).. Sine? all Itemtf appearing 
on Forms-i- ar\d 3 were tjngroupell, the tJIrectlon'of differendcs between th^ 
observed correlations and the testes of statistical significance for 'd Inference 

. proyrded for an assessment of a sequence, effect apart from an Item grouping , 
effect. Although possjbly subject to a. group I ng, effect , add l^onal Vcomparl ^ ' 
sons werA made for adjacent vs. nonadj-acent Item pairs on^'the other forms/ 
^ - Jhe second procedure Involyed th^ of COFAMM. a computer program for 
confirmatory and exploratory factoV analysis (Sbrbon. and Jdreskog. 1976). ^ 
It Mas used to test (1) the equlvajence of the coviarlance matdces for" 
particular pairs of questionnaire forms. (2) the^models- specified for thfe 
cdVaMance matrices af ter.appropr late equality constraints had been relaxed, 
•nd (3> the fit of a factor solution for Form 1 (the org I naJ- questionnaire 

• fofm) .to comparable data obtained through other fbrms. . Useful teferenccs 

• ■ 

.on the use of this program are the COFAMM Users. Guide, Jorskog (1971)^ Long » 

, 0^76). and Sbrbon (197^). * * • 

• • • " "-r-^" ^ 

, In general, the, model tested by COFAMM, and Its associated parameters 



where ?$ a p x I vector of observed* values Jor group g - 1 or 2, Yg \s 

« p X 1 vector of- loqatlon parameters, A ' Is a p x k parameter matrfx of 

f«ctO|^ loadings, fg the k common factors, and being the p unique factors 

or residuals'. .If It Is assumed In the model that E(zn) - 0, E(f ) - 0 * « 

\ -9' ^-g' **a* 

•nd 2 and f tof'be uncc^rrelated* It follovjLthat; 

( . r ^ 



0 
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Mg - Yg + AgGg - . v (2) 



and that , ^ . 

' - Ag*gA'g'+ 1% "\ ' ' . (3) 



• V 



where fs the covar lance ^matrix of fg,-1'g the covar.lance matrix of 2„, and 
y f$ the mean vector of x-,. 



In using COFA^M, One .has three options In specifying characteristics of 

» • # • « 

parameters In a hypothesized model:. (1) fixing parameters to haye assigned 
valties» (2} constraining parameters which are unknown to be equal to one or 
more other parameters, and (3) freeing parameters which are unknown and not 
cons trained to be equal to other parameters. ,|f a hypothesized model docs not 
provide an adequate fit to the data, as determined by the chl-square statistic, 
then some of the fixed and/or constrained parameters can be relaxed (freed) 
«nd, a nj^w chi-square statistic for the modified model Is computed. In 
general 'there li Interest In both the adequacy of fit for the newly specified 
model and- the reduction In the original chl-square s^lstlc/brought tfbout 

^ relaxing the constraint^ on the parameter.s. v ' • 

\ Jh' testing the equivalence qf pairs of^ovarlarice.matrlces, Aj and A2 ' ^ 

.were constrained to be equal p x p idei^tlty njatrlcb^, %\ and ^2 were constrained 
to be^ equal p x p matrices, and j;i anjsl 4;2 were fixed to be equal to a diagonal 
matrU,* but contain Inferos on the diagonal (Q) . When the chl-squarev^tatlstlc 
Indicated a lack of flti equality constraints placed on^g were relaxed to 



t«it hypotheses about order and/or grouping effects. Parameter specifications 
for fitting the Form 1 factor solution to the correlation matrix of oth<»r 
jrogps were to fix to equal A], free « and free a diagonal This pro- 
cedore Is consistent with the ones Illustrated by Sbrbom ariW Jbrcskog (1976) 
and Lawly and Maxwell (19Z1), but they simultaneously fit ^ groups with 
6.) * ^2' ?J lit !l !2 specified as free dlagopal matrices. Selected 
drfferehceJ between the observed and estimated matrices, the residuals, were 
rnspected .to determine If a pattern reflecting ao Item sequence effect could 
b«r Identified. ' ' , 

Factor analysis procedures were also employed In an exploratory sense. 
A princtpal components analysis was conducted a4-a means for determining the 
numbep of factors needed to explaFn the variation In an Item set for the 
Form 1 data. This analysis was conducted usjng the Statistical Package for 
' the Social Sciences (SPSS) factor analysis routine. • information used In de- ' 
trdtn'a on the number of factors was the determination of the number of elgen-" 
values, greater* than one, the relative values for all eigenvalues (Scree test') 

and the l^nsp6ctton of Varlmax rotated solutions corresponding to Iterated 

.t 

prtnclpaJ factor analyses based on a fewer number of factors. 

Using the ryumber of factors determrlned from the SPSS ahalyslsj the • 
computer program EFAP (Jbreskog and Sbrbom, 197.6) was used to generate the 
factor loading matrix used as Input to COFAhW. A decision was made to- pro-' 
duce the factor solution using the unweighted ledst squares (ULS) option for 
.^FAP as this produces solutions equivalent to those obta Inedy^rom the Iterated 
prtnc I pari factor method (see EFAP user's guide) . • 

Since the typical researcher Is concerned with result^ of factor analyses 
for exploratory and/or data reduction purposes,, defaul t opt l ords ' for the SPSS 
feptor routine were used. The procedures for obtaining the factpr soliftlon for 
each ot Forms 2, 3, ^ and 5 were cohsUtent with that obtained for Form 1. 



Th«t# five resultant sets of factor loadings were .^en> Inspected to pscertain 
whether differences lo the consJtructl9n of the quest tonn tare affected the re- 
suits and, consequently, affect how one might proceed In reducing data by 4. 

g 

summing ^he associated Items for the Identified factors. 

A limitation to the study was that the real core requirement needed to 
proceajj} analyses Involving all 61 I terns exceeded what was available on the 
local computer configuration. Consequently, the analyses were performed witl) 
four subsets of Items rather than with the entire set. Differences In^the 
Item organization of Forms 1 and #'was used as the basis for determining the 
subsets. The subgrouping of I terns are Identified In Table 1. 



V 



Insert Table 1 about here 
u_« . 



Item Subsets 1 and 2, each containing 17 Items, were of 'most Interest 
since the primary differences between Forms 1 and ^ were the result of re- 
ordering and recouping these Items. Subset 1 contalfTs*two sets of Items, 
10-13 and ^0-^3, which appeared as separate groupings on Form 1 , but on ) 

Form k each set was combined w ft h other Items. Items 10-iywere combined 

/ \ 
with Items 16-19, and 2^, and Items kO^M} were combined with 19, 22, 

*" 23, and 25. Subset 2 appeared as a single group of Items on Form 1 , but ^ 

V 

was split Into two Item groups on Form k: Items 32.-39 an* 27-31, An 
additional modlflcatj^n fgr this subset was to g^up Uems ^, 20, 21, and 26 
with Items j27*3l 4/ 

^ Item .^iubsets 3 and k appeared Identical Iv/^quenced on Korms I and 
Consequently, tests of* .the equivalency of the Form 1 artd Form k covarlance 
matrices were expected to display an 'adequate fit for SubsetV 3 and k. If 
Item grouping or sequencing effects were opera'tlng, differences between Forms 
I and k should be expected for Item SubieU I and 2, ^ , 
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Results Indicative of the sequencing and qroupfng effects follow. • 
' Equtvaleace of Forms 1 and 2» Forms 4 and 3, Forms 2 and 3, and FormV 1 and 
H would fndlcate neither Item sequencing nor Item grouping had an effect. 

V 

Orfferences for all four of the comparisons would Ir^dlcate both Item se- 
quenctng\nd Item grouping had an effect. Since Items on Form 2 appeared 
tn the same order as on. Form 1» but were, ungrouped» a result finding the. 

^^^"^^ I ^» ^l^^*'"«"ces<?between^orms 1 and 3 (the 
random sequence form),- Forms 2 and 3» arid Forms 1 and' 4 (for Items reordered 
on Form A) would Indicate only a sequence effect.. Differences between Forms 
1 and 2 and Forms 1 and (for^ I terns reclassified on Form but not re- 
ordered) would Indicate a grouping effect. 

' . Results ! ' 

Corre lations . , 

The Pearson Pwdnct Moment Correlatlons/fb'.r adjacent htem pairs on 
questionnaire Form 3 are provided < triable 2. , ~ 



Insert Table's about here 
— 

'V. 



Since Form 3 represents the random for*m and the Intent was to seek tonf Ir- 
metlon that Items appearing as adJ.^Qent Items would correlate to a higher 
degree with one another than If the Items Were »sepa rated, comparisons were 
made between those Item. pairs appearing Ss adjacent or^ Form 3 but no.t on 
Form U The differences between corr^el at Ion's were then tested for statls- 
tlcal significance. A twor-talled test with an alpha tevel of .oi was used 



as a partial control for the rtrge nufnber of tests made. Of^ the 61 Item- 

pah* correlations reported, 21 were statistically different with the higher 

correlation always favprlng the adjacent Items. Of the remaining 39 Item 

pairs 3^ observed differences In correlatlqns were In "the anticipated direc- 

tlon while only 5 were In the Opposite direction'. -It should be noted that 

although there weV6 5Jtem pairs for which th^observed nonadjacent. appear. I (ig 

Items, correlated more highly, the d I f ference wl fih tfie largest magnitude was 

only -.062. Also^ It Is Interesting to note thaJt for k of tl^e 5 pairs where 

nonadjarcent Items had the larger ojbservcd^<iorrelat lon» the nonadjacent Items 

appeared withirr the same hdentlfled group|of Items. 

>^port fo^^ above findings ^s contained In the Information associated 
■ " - \ • 

wJth the Item M^s as determined .from Form\2 shown In Table 2. The Sterns 

on. this form are In the same sequence pis for Vorim l but .do not appear In 

Identified groups. In comparing the K>rm 2 anoXForm 3 correlations It. Is 

seen thait^A out of 60 differences were statist I caMy greater than zero with* 

58 observed differences greater and on 2 observed. differences less than 

zero. It Is $^'9n.lf leant that the^la'rger of these negative differences Is, 

In an absolute sense, only .096^ > ■ - » ^ 

Vanjance r Covar lance Equivalence ! ^ ^* 

* • ■ * 

A summary of *lnfci,rmatlon obtained from the comparUons of the covar lance 
matrices Is provided In Table' 3. 



^ — • • Jl^-. ; : 

Insert Table 3 aboMt here ' . -/ 

** ^* ^ •» Mi «• M «B «B w as tm «• 

f • . 

■ .\ ■• ... ■ • • 

Uijp9-^n alpha level of ,01. I.t j^an be seen that for Item Subset 1 Cltems 10-lA. 
16-19. 22-25,' and AO-43) the results marginally Supported a model assuming 



n 



•quIvaUnt covar lance matrices for Forms 1 and 2. (x^ - 201.18, df - 153, 
and p < .0060). Th|s would. Indicate thUt grouping l^ems for the respon- 
dents majy not necessarily Influence the covarlances of It^ms app«{arlng 



"l^n a que^tlonnaJre. 



For the seme set of Items a comparison pf Form 1 responses v/^th Form 3 
(random) responses Indicated a def In Ite Jack of fit to the hypothesized model 
of equivalence (x^ - 393.00, df - 153', P .< .Oobl). Thus, some evidence for the 
suspacted Item sequence effect was apparent, sjjpport for this finding was found 
when the equivalence of Form^2 and Form 3 matrices was tested and the result 
was again evidence of a lack of fit (x^ - 395.09, df - 153, p<i*0001)^ 

The comparison of Forms 1 and 4 for item Subset I was of special Interest. 

V 

I 

The regrouping' of these on Form 4. resulted Tn a drfferent Item sequence than 
on Form 1 for somerftem paUs but not^thars." Based on the results reported 
earlier for comparisons of correlations of Items adjacent ort one fl)rm but 
nonadjacent on a second. It was expected that a difference between the- two ' 
covarlance matrices could be partially explained 'by dl|ferences between co- 
variances for Item palrsSequenced differently on the tvyo forms. Thi two 
matrices, shown as correjatlbn matrices, appear In Table Those 'pa1 ry of 
Items whose covarlances were expected to be Influenced by a sequence effect 
ai'e Indicated by numbers used as superscripts. It shouH be noted that all 
differences between these pairs of Items were In the anticipated direction. ■ 

'^ 



insert Table k about here 



Correlations for additional adjacent Item pairs ^re presented In Table 5. 



* * « 

- ^ ^- \ 

Insert T*ble 5 about here 



» 12 
Th€S« pairs are composed of one Item,from the subset and ^n adjacent I tern 
which was* not considered to be part of the subset. The Item pairs pre- 
scinted are limited to those that appear as adjacent Items on only one of 
• Forms J^and 4. For Item Subset 1, all eight of the Form I adjacent Item pairs 
had higher correlations than for /orm A. However, only one of the throe ^ 
•d^QCcnt Item pairs for Form k had larger correlatlons"\han found for Form 

Hesul ts, shown In T^le 3 Indicate the equivalence model for Item 
Sul^et I did not fft the cohiparlson of Forms I and k (x^ » 280^18, df - 1 53, 
- ^<.00O1). A^econd model In which the equality constraints were relaxed 
for parameters associated with adjacent Item covar lances also provjded an 
fn«dequat©*'f It to the data (x^ - 199.68, df - |i»5, p<.0020). Thus, It 
appeared additional differences existed between the two covar lance matrices 
nop taken Into account by the newly^eclf led model. However, the relaxa-: 
tton of the ©quality constraints for these cgvar lance parameters did provide 
for a Significant lm|?rovoment In the fit as demonstrated by the significant 
reduction of the chl-square value (^2 . 8O.5O, df - 8, p<.O00l). This re- 
lUlt provided additional support for an effect of Item sequence on the co- 
variances correspond I ng. toVadJacent Items. ^ ^ 

Since the effects of Item grouping and 'sequence coyld be confounded for 
additional comparisons of Items grouped differently orTxprm k and 
Form 1 (Items 16-19, 22r25) » i^dltlonal models refJectlng a distinct 
grouping or sequence effect were not^tested. Instead, a modal was specified 
which retained the equality cons^lnts for only (teitis appearing In the 
tame groups and In the same sequence on both forms (Items 10-13 and 40-^3). 
Thii modal provided for ^^gnlflcant reduction In the chl-square value - 



(x^ - .170.27, df - ffia, P < .0010) and an adequate fit for the data (^2 -^29.40, 

df - 36, p < .7739). Thus, the reorganization of the questionnaire affected 

only those Items which appeared* In different sequences on the two forms. 

Although a model was not specified to test for a second order lag 

effect, (I.e., on© Item separating an Iteri pair) an inspection of the 

♦ *■ ■ ■ • , ' 

Form I and Form k correlation matrices reveaJed that all of the nine pos- 

sfble comparisons of second order lag correlations favored the , form on which 
thtf ftei1*palr appeared with only one Item separating the pair. These nine 
correlations ar^ indicated In Table k by alphabetic superscripts. 

Forms 2 an^ 3 were uied In analyses for testing possible second (one 
rtem sopara^on) and third (two' Item separation) order lag effects In 
^ddltlon to testing a first order effect. These forms were used for the 
analyses because the''lt;ems appeared In a different sequence on the 'two forms 
and rtem grouplngs/i/ere not employed for either form. Consequent^^Mtem 
grouping and sequincTngA^ere not contended. 

As shown In/Table 3,\a. m^el^or which the first order equality constraints 
were relaxed In the Form 2\and Form 3 comp^lson provided for a significant 
reduction tn the\ibhl-square*vaTlue- (x^ - 105.13, df - 13, p<.POO'l). Although 
the chl-square value was also significantly reduced for models relaxing the 
second order equality constraints (x^'- 21.6A, df - p^.0110) and the third 
order constraints (x2 - 44.42, df - 8, p <.0001),\the relaxation" of first, 
seconfd, arid third order constraints did not provide an adequate fit for the 

deta Cx^. - 223.90, df - 123, p<.0001). This ^tfUggests that other, unexplored 

'i 

differences existed between the two data sets. An lnspectloi> of the correla- 
tlons for Form 1 and Form 3 revealed tbat the most consistent Influence of 
ftem sequencing wa^ Its Inflationary effect on correlations between adjacent 



ftem$« ^ If an Item seqilcriclng effect. wgs operating on second and third order 
correlations, It did not appear to be as' >yst4mat Ic as for adjacent pairs. 

The same procedures were used In the analysis of Item Subset 2, Items ^ 

20, 21, and 26-39- The distributions of^ Subset 1 and 2 I turns were slml^ 
lar In that Sublet 2 Items also appeared as two separate Item grouf/tngs on 
Form ^ and gne or more of these Items appeared wl^hl^onc of three Item 
groupings on Form 1. Items 27.-39 appeared In the same seq.uence-*bn both 
^ormsi but Items 32-39 were placed In a sjacond grouping on Form h. 

Results for the ov^erall comparisons for Subset 2 (displayed In R&rt 
B of Table 3) were similar to those found for Subset 1. Models for which 
equality constraints were placed on all parameters of the covarlance matrices 
yielded significant chl-square values for comparisons of Form 1 with l\orm 2 
(x* - 2U.27» df - 153. P <-.0008)» Form 1 with Form 3 (x^ - 358.72» df^V ^ 
153, P < .0001) » Form 1 with Form k (x2 - 237.85. df -.153., P < .0001). 
and Form 2 with Form 3 Cx^ - 337.30. df - 153. P < .OOai). As with the 
analysts of^ the Subset 1 Items» the smallest chNsquare value was obtained ^ 
from the comparl'son pf^Form 1 to Form 2 and the largest chl-square values 
resultud from the comparison? of Form 3 with Form 1 and F'orm 2« 

* Results for Item Subset 2 analyses based on modified models^ wherein 
equality consti;iaInts were relaxed^ were also similar to those found 
for Subset 1. Itelaxatlon of the first order lag constraints for the Form 1 
with Form ^ comparison resulted In a sl^filf leant /eduction In the chl-:square 
value (x^ - 9.80. df - 3. P <-02^9). The correlation matrices- for both 

< 

forms a ra provided In^Tablc 6. Al l three of the dl ffer«inces . between the 
adjacent^ I tarn pa^r correlations were In the anticipated direction. These 
correlations are Indicated In the tabl^^ numerical superscripts. Also, 
four of the five differences for Item pairs with one Item appearing between 




15 

th«m were fn the •ntlcTpated direction. Tjiese are Indicated. by alphabetic 
superscripts. j I . 



insert Tabic 6 about here 

Additional adjacent Item pair correlations (one Item of the^palr not a par.t^ 

of the set) were also computet^ for Subset. 2. Three of the four Item pairs''^ 

- » * 

Identified In Table 5 had larger observed corrcHatlons for the adjacent Item 

The mod6l In which Equality constraints for parameters associated with 

V ' ' 

4toms no^ contained In the same Item groups In both form^ (Items 20,. 21 , 26) 

were relaxed, yielded a significant reductlcSn In the ctil-^square value (x^ " 

116.20, df - 59. p < .0001) and provided for a reasonable gobd fit (x^ -111.85 

df ■•.91, p < •0681). A hypothesis .of Aquallty of the Form 1 and Form 4 co- 

vertiince matrices f^r Items 27^33 could not, therefore, be. rejected even 

though 27-31 and 34-39 appeared In different groupings on Form k. Thus, 
• ^ f » * 

as found forNltem Subset J, the major effect'of reorganising the question- 
• ilAlr# appearedNto be riiflecte^ In the covaV lances Involving those Items that 

\ • . . . I ^ 

appeared I n a diffeT'ent sequence* . * ^ 

Subset 2 chl-square values wer^ sign If leant In the Form 2 and Form 3 
comparison for firs-t orc^er (x^ - ZkB.SJf df - J39, P < *0001), second or- 
der (x2 • 240.77/ clf - 127^P < .0001), and'thlrrd ord*r (x^ - 230.67, df - 
116, p < .0001) lag effects. • The model relaxli)4 the equality constraints on 
frrst order. Ug pafr« did provide for a significant reduction In the chl- 

tquare value (x^ - 88.73. df - l^*, p < .OOOl^lbut models relaxing con- 

' /I/ ^ ' - 

s^jF^Int* for second cirder^(x2 - 7.80, df ' P < .8000) and third order 

(X^ - 10.10, df - 11, p < .5200) lag effects If I J not result In a slgnlfl- 



cent reduction. This finding, the relatlv^ m^gtlltude of the reported chl- * 
tquare velues, and an Inspection of the twci corfalatlon matrices Indicate 



• pattern similar to the one described for Subset 1 Items, kat Is, an . 
InfUtlonary effect >Df Item sequencing was operating and It was fpos^-«ys- 
^ temattc for first order lag correlations. 

Results for Item Subsets 3 and k were, '^s ant Iclpated , quite similar. 
The, Jtetp organ^zatton foi* these two subsets differed from that for Subsets 
\ and/ 2 In that ItAms In Subsets 3 and '^•appeared on Form k In the same 
^quence and groupings as on Form 1., Assuming the dJ^a to be consistent, 
the results for Subset** 1 and 2 suggest one vvou Id expect differences to b« 
found between Forms 1 and 2, Forms I and 3, and Forms 2 and 3. Because no * 
differences between Foilms l and >^ wer^ found In the previous Analysis Ijn- 
vblvlng Items 10-13 and kO-k3 In Subset 1 and Items 27-39 In Subset 2 (those 
r tcms* appear I ng In the same sequence for. both forms), no differences were 
expected betweeh Forms 1 and k for Subsets 3 and k. 

The chl-square valuHs (see Table 3, P^rts C and D) f^^ltlngvfrom the 
comparisons of Forms 1 and 2 with Form 3 were significant for both Sqbset 3 
(x2 r 501,7^. df - 171. p <-.00Ol and - ^71. '36; df UK p < -0001 , 
•respectively) and Subset ^ (x^ - 7A,8l, df - 36, p < .0002 and x^'-** 
82.10, df - 36, p«<.0001, respectively). As was antlcjpated, the chl- 

square values resulting from the comparisons of Form 1 wtth Form"4 *for 

* \ 
Sub»dty3((x2 - 201.96. df - 171. p < .0528) and for Subset k (x2 - 

39/^3, df - 36. p < ,'3913) we^e not significant. The only deviation of 
the findings for tihe$e two subsets from resijlts reported fjr the other Item 
subsets was that the comparison of Form 1 with Form 2 was significant for ^ 
Subset 3 (x^ - 231.87. df - 171. p < ,0013) but not sfgn If leant for Subset 
*i Cx2 - A2.A8. df - 36. p, < ,2119). 

• Resul ts Involving the possible Item s^uence' effect on the first, .se- 
cond, and third order lag covarlances Indicated the same patterjn ^ was 
found for Subsets 1 and l\ The largest reductions In the chl-square values fo 



tt^m Subs«,ts 3 and k were those Involving the flrs^t order lag effect (x^ ■ 
176.58, df - 17. P < .0001 and ^26. '♦7, df - 6,^p < .0050). A third or- 
d«r Ug modal did not fit the data for olthpr Item Subset 3 or Subset The 
reported tendency for first order lag correlations to be, most affected by 
tequenctng was also observed In the Form 2 and 3 torrelatlon matrices for > 
both Subsets 3 and h, ' 

Factor Analysts 

, X ■■ . ■ 

.Prior to fitting the Forfn 1 factor m^rlx to Form^ 3 and It was \ 
first used fn an analysts of data collected previously for a sample of 
teachers. Since the teacher data were collected on a fo^-m Identical to 
Form 1, the analysis provided some evidence* of the'stabll Ity of the factor 

matrix. For purposes of this analysis, a sample size of 257 was declared 

■* K 

tn. order to obtain a chl -square comparable to that for Form k. The analysis 
resulted In a nonsignificant chl-square (x^ - 560.32,. df - 5^»0, p < .2640) In- 
dicating the factor structure was reasonably consistent. for aft Independent 
repUcatlcin. ft Is of Interest to note the obtalne^ chl-square value apprt)xl- 
mated that \>b tanned fbr Formr 1 (x? • 529,03). 

The cbl~square^ values obtained In fitting the Form 1 factor matrix to 
Jorm 3 (x2 - 1235.70S^df - 5^0, p < .0001) and Form 4 (^2 - 1255.30, df - 5^0, 
P < .cool) we|[e both significant. Residuals resul ting ^rom these wo analyses, 
the teacher data analysis, and the Form 1 data analysis are shown In Table 
7, Because residuals produced by COFAMM.are calculated by subtracting ele- 
nien|pof the sampl^e matrix from elements of the matrix estimated from thp , 
model, ntsatlve residuals Indicate underestimates. 

■i- t-.. 



Insert Table 7 about here 
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If Jtem sequencing can affect results ,of factor analyses, the resi- 
dua|s recorded In Part A of Table 7 sho(ii^d be more positive— Indicative 
of over,e$tImates--for Form 3 than for the teacher, data, ^rm 1, or Form k. 
that Is, the Items. listed In Part A were In the same* sequence on all » 
forms except Form 3, thus, analyses using the factor mc^tr I x .obtained "from 
Form 1 should prov Ide . slml lar residuals for the teachec/fYf^'*'" ^ » 3"** ^^rm ^■ 
k data, but pyerest Imates for Form 3. Of the 23 ltem^)alrs Identified In* 
Part. A of Table 7, -17 of the Form 3 residuals. Indicated with plus signs, 
were more positive than the residuals for the teacher. Form 1, and Form k 
data. Of^the remaining six pairs, four of the Form 3. residuals were more 
positive that) two of the^pther three groufh. . ► 

Residuals shown In Part B ot Tablff 7 are fo^ Item pairs w/lch appeared 
adjacent on Form but nonadjacent on the Teacher form. Form 1 and Form 3. 
Consequently, the residuals would be expected to b^ more negative— Indicative 
of underestimates— for Form k since the factor mat r ix *flita Ined from-Form 1 ' 
data would not be Influenced by the proximity of the adjacent Item pairs. 
Eight of nine res^duals for Form k were more negative than those ftf the 
teacher. Form 1, and Form^ 3 data. For the one Item. deviating from the pat- - 
tern, the Form '♦'residual was more negative than It was for ofte qf the other 
thA^ groups and i^Iffered from a second by only .OO9I 

Item pairs shown In. Part C of Table 7 are those for which the'resldual s 
fhould be more positive — Indicative of overestimates — for both Forms 3 and k 
stnce these pairs of Items, except I hg Itemsjj^ and 32., were adjacent on the 
taachar data and Form 1 ,'vt)gt not on Forms 3 apd k. Items 31 and 32 were also 
adjacent on Forin 3. As shown jn Table 7, the residuals were more positive 



for both Forms 3 and k for sIk of the .seven pairs which was consistent with 
•xpactattons. Additionally, the residual of Forjn 4 was more positive than 



/ ^ tht residual s of teacher data and Form 1 for ftem pair 18 atld 19. the Form 

h residual was also more, posit Ive for lteni>alr 31 and 32 than those for the 



other three forms. ^ ^ 



Item pairs shown In Part D of Table 7 are thos^ for which the residuals 
should be more negative— Indicative of underestI1^ates--for Form 3 sirtce these 
, Item p&Irs were adjacent on Fprm JTTbut nonadjaccnt, on the three other forms. 
, fourteen of the seventeen residuals were'more negatl^e^ fAr Form 3 than for 
- the other- three forms. , - ' ' 

Overall, simultaneously cons IderTng the expected relationship among all 
four forms for each set of Item^ pair residuals, ^6 of the 57 werp In the Igf 

pected direction If, a sequencing effect was operating. Considering palrwise 

% ... 
I differences, 151 of ,178 were In the anticipated direction. " . 

Exploratory Analyses { , 

Results f/om the Varlmax rotation of a" six- factor solution for each of 



the four forms are shown In Table 8. 



--Sfu... 

Insert Table 8 about here 



the l^emi^'^e shown as they were grouped iind In the sequence they appeared 
on Forms 1 and 4,. The four columns under Form 4 IniJIcate the four' Item 
,grouplngs.dn Farm%^ The setslpf rows at the left of the table Indicate 

^ the Item groupings on Form K When possible^ the most stmfl^^ factors w^re 

> * ■ . 

assigned the same fajctor numbers. Only the largest factbr loading for each 

■ « ' \ - 

Item Is shown. H 

. . .■ ' • . ■ c ■ ■ 

The findings from the comparisons of the covar lance nta trices for the 
, four forms suggested that (1) the ^factor matrix fpr Form 3 would deviate most 

er|c : ■ - -^S. . " • " . 



. " r ' ^ .20 

from the factor matrices of the other three groups sInX the large^f ctil- 
»qUare values were ot>talned \n comparing Form 3 to Forms 1 and 2, (2) the 
factor matrices of Form I and Form 2 would be mo^t similar slnc^ the smallest 
chl-squar^ values ,were found In comparing the Form^ find 2 matrices, and (3) 
the factor matrices of Forms I and k would deviate most for .Items A, U. and 
16-26 since analyses with a modtfled model Indicated 4hat the difference be- 
tween the two forms resided with these particular I tarns. 

shown In the table, the factor matrl^j^of form 3 'did deviate most 
from the factor matrices obtained fo^r tV other three forms. '.Also, the fac- 
tor matrices of Forms I and 2 seemed most similar; with the majpr differences 
betns a shift c^f the adjacent item pair of 16 and 17 from factor 11 on 
Form. 1 to the sequencq pf .Items 18 to 21 , factor ^ll, on Form 2 and (2) a 
shift of the adjacent Item pair of 27 and 28 on factor II of Form 2 to the 

sequence of Items 29-31 » factqr III, on Form 1. While, overall,, tWe differ- • 

t 

ence between Forms 1 and k does not appear to be much greater than between 

Forms 1 and 2, It Is significant that most deviations of , the Form k factoT^ 

matrrx from the Form 1 factor matrix resld^^lth Items 4, H, and 16-26. 

Additionally, these deviations correspond to the grouping and sequencing 

used on Form k: Factor 1 of Form. 4 corresponds ^o the Form k Item grouping 

!, Fac^tors II and VI correspond to item gpjuplng 2, Factors III and IV cor-, 

respond, to Item .grouping 3» and Factor V qorresponds to Item grouping A. 

WhMeil^rm. 5 had not been Included l<i any of the previous analyses — 

primarily, since nine Item groupings were used on the form It added to 

'. * ^ , ' • ■ 

the complexity of the study— It was decided that results of a factor analysts 

« * » • * 

of Form 5 might provide confirmation for the r^lts of the factor analysts, 
particularly those for Form 4. Since Form 5 represented yet a fifth way In 
which the Iterirs were organized. If quest lonna Ire organ Ij/^t Ion Influences the 



• / 

tnter-ttem cprrelatlons «nd subsaquent factor analyses, results from Che 
factor analysis ,would be expected tp reflect the organization of ttie Items 
on Form 5. >y v ^ 

'.The Item sequence and-grouplng and the results of the factor analysts of 
Torm 5 aVc shown In Table 9. Si^ome llbprty has been taken by showing two 
loadings /or Items 11, H» and 26; however^ the addition pf these second 
highest loadings could not be'roslsted In light of the very close corre^-^ 

♦ . . . . ■. .... . a. * - . . 

pondence of the first four factors and the Item groupings. 

I . 



Insert Table 9 about here i 
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Wktle It could be argued that only four factors are present, the sixth 
eigenvalue for Form 5 wa's 1,08 and^ th© relative magViltudes of tho values 
might Justify six factors, Regardlessji^ thft^anal"7sls does provide additional 
4vldence^of the Influence of questlonnalr.e^organlzatlon on a result€ht fac- 
tor matrix for these Items, 

Summary and Discussion 

t 

The results of the comparisons of Pearson product-moment -Correlations 
provide. substantial evidence In support of adjacent Item pairs having In- 
fUted correlations relatlve'to what one vyould anticipate If the Items had 
not appeared concurrently on tbe questionnaire. In genera^ the cov*irlance 
matrices corresponding to particular sets of Items for two quest lonnlare 
forms composed of grouped Items were found to differ If Item pairs were In 
« different sequence, and not. to.d If fer If the Items appe.ared In the same 
sequence. Improvements In fitting data wlth a model assuming' equ I valent j 



, • . ' 22 

cov«r?ance matrices were generally found by specifying a model allowing 
covarlances for Item pairs adjacent on one form but noaadjacent on the se- 
cond form to differ, but requiring Items appearing In the same sequence on 
both forms to have equal covar lances. Comparisons of two other questionnaire 
forms on whlcii Items appeared In different sequences and ungrouped Indicated 
that although sequencing had some effect on second and third order lag co- 

variances, these effects we>e less systematic than the effect of sequencing 

, , .... 

on first order lag covar lance:; (adjacent Item pairs).. 

An examination of the residuals obtained In fitting Forms t» 3, and k and 
the teacher data using a six factcr solution obtained for Form I data provided 
evidence that item organization could affect the factor structure. Differences 
observed amojT^the >f ac^tor structures of five different^ Item organizations re- 
sulting from'^hc use of rather conventl-onal factor analysis procedures 
reflected these different Item organizations. The differences among the 
factor structures could be attributed to the Items which appeared In dif- 
ferent sequences on the forms. - 

Although *^ome evidence was. found to suggest Item grouping with an 
associated group descriptive caption had some effect on th6 covarIances» 
tt was difficult to draw firm conclusions about the strength af«»uch an 
effect. Comparisons In>S^lvIng Items that appeared '{n the same sequence, 
but were grouped on one form and ungrouped on another resulted In the 
sndllest differences of all the comparisons made In the stUdy. The results 
from exploratory factor analysis procedures Indicated the grouping effect 

• \ 

was quite small. If even present, at leas^ln comparison to the sequencing 
•ff«ct, , 

The most striking result of the various analyses was the consistency 
of findings, Tha affects of I tem.sequencje wece- general ly present when anti- 
cipated and absent when unanticipated^ 

> 95 



Jh« findings reported here are of course restricted to data of the»type 
d«tcrtbed. However, this type of data Is representative of a vast array of 
data used by researchers. Researchers working wlth.t.hls type of data who In- 
clude^factor st^ructure related procedures as part of their analyses migbt 
consider a possible Item sequence effett In Interpreting their results and In 
constructing their questionnaires. 

\ .... 

V The question of the presence of an Item sequence effect In the analyses 
of questionnaires containing high- Inference type I terns Is raised by the re- 
sults found In this stjudy, Such measures are general 1y derived and/or con- 
.firmed with factor ai^alysls procedures.* Some consideration should be given 
to asking If the same factor composition woujd have resulted had the I terns 
bMn presented In some other sequence. Atten^J^n^houl d also be given to 
factor solutions of only a few Items. It Is po«lfle such factors ^re only 
reflecting Inflated relationships resulting from Items of close proximity.- 
Finally, one might hypothesize that a sequencing effect. would be less 

Influential on analyses Involving fewer numbers of factors. This Is cur- 

* • ' ' ' ■ , • • , 

*rintly being Investigated using the data obtained for this stud*/. " 
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TABLE 1 



ITEH SUBGROUP COMPOSITION FOR THE 61 ITEMS APPEARING 
ON THE IN-SERVICE EDUCATION SURVEY 
* . QUESTIONNAIRE 



Subset 



I tern number 



Form 1 



Group 



l| 21 3 



Form k 



Gro^p 




10 

11 

12 
13 

16 

>7 
18 

19 
22 

23 
2k 

25 
ho 
k] 
k2 

JH. 



20 
21 
26 
27 
28 
29 
30 
31 
32 

3? 
35 
36 

37 



X 
X 
X 
X 



X 

X 

X 

X 

X 

X 

X. 

X 

X 



X 
X 
X 



X 
X 
X 
X 



X 

*x 

X 

x 

X 
X 
X 

X 
X 



X 
X 
X 
X 
X 
X 
X 
X 



X 
X 
X 
X 
X 
X 
X 
✓X 
X 



X 
X 
X 

X 
X 
X 
X 
X 



X 
X 
X 
X 
X 
X 
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TABLE 1 --Continued 















* 




Subset 


1 tern number 


Form 1 


Form k 






Group 


^roup 


X. 




1. 2 3 





kS 

47 
48 

49 
50 
51 
52 
53 
54 
55 
56 
57 
58 

59 
50 
61 



X 
X 
X 
X 
' X 
X 
X 
X 
X 
X 
X 



X 
X 
X 
X 



X 
X 
X 
X 

k 

X 
X 
X 
X 
X 
X 

x^ 



X 
X 
X 
X 
X 











1 


X 


X 










2 


X 


X 










3 


X 


X 










, 5 


X 


X 










6 ,^ 


X. 


X 










7 
8 


X 
X 


X 






\— 




? 


X 


X 



Note. --A1 though Identified as Item groups 1, 2, or 3, each subset 
consists of distinct groupings, I . e. , Items In groups 2 for Item subset 2 
•re not necessarily In group 2 for Item subset 1. 
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'TABLE 2 



?8 



PEARSON PRODUCT MOMENT CORRELATIONS FOR ITEM' PAIRS 
APPEARING AS ADJACENT ITEMS ON QUESTIONNAIRE 









FORMS 


TWO AND 


THREE 






1 


Forin 3 


Form 1 


r 1 III J 


3-1 


rorm z 


ro rm z 


roriM J 




pa t rs 


N»480 


N«?54 






N w 2ii 1 








075 


179 


1 04 


1 2 

1 9 z 


1 76 


zpH 


U/o 




259 


2 1 1 


-n46 


2 1 


liiC 


19C 


no f\ 
UzU 




208 




0 17 


\ ii 


7 1 1 


9 Q 9 
zjz 


039 




429 


517^ 


1 OR 




C 1 9 


iil 7 


nor 


21 I 1 


l44 


166 


n99 


!>i ^ 


**OD 


**pb 


01 0 


1 1 ^2 


176 


219 ^ 




6 7 


• 977 
z// 


1 Co 


no 1 


42 kO 


196 


114 


*062 
UOx. 


7 ft 


iiiiii 


999 

ZzZ 


0 0 OW" 

zzz^ 


iio n 


255 


286 • 


OAI 
UO 1 


ft Q 


' A eft 


iiA 1 
•♦0 1 


197* 


13.15 


115 


44l 


1 2ft 


Q 1 n 


99 C 

zz^ 


9 0 1 


099 




129 


li42 


0 1 1 

n7i 


in 11 


0*40 


iiAi 


1 o3*f 




111 




11 1 9 
M 9 1 z 


ooO 


iiCZ 
**I)0 


20*** 




1 1 c 


inn 


O05 


12, 1 3 




0 *i r* 

235 


399* 




1 1 A 








iiiin 


323 


VI7 




liiO 






1 ii 1 C 




Z\J/ 


> 0 Ji 0 X 






161 


1 9 1 




Q oZ 


9 AO 

zUz 


1 9** 




216 


•f H 0 


979* 


1 A 17 
1 0 9 1 / 


b/b 




0 1 AX 
219* 


61 20 


218 


269 




.17 1ft 




ill ii 


1 r 1 
151 




261 • 




1 1 1 


1ft 1 Q 


c 9 c 
bib 


i*90 

Ijz 


1 A 0 

103 


^2 11 


4 08 


MOO 


n7fl 
\j / 0 


1 Q 9n 
1 ^9 ZU 


C C7 




1 

157 


31 27 


507 


611 

0 J J 


' 126 


20 21 
Z U 9 z 1 


61 1 
oil 


m6 


075 


27 38 


111 


110 


-*00 1 


21 22 
^ z 1 9 zz 


^7n 


111 






1 24 


107 


1 fll 


99 91 
ZZ 9 Li 


ii 1 A 
*♦ 1 0 


1 1 ^ 


1 nn 
1 UU 


181 


4l6 

M JO 


2 I^IA 
z 


91 9ii 
Z J 9 Z*f 




9nA 
zuo 


Q it 0 4 




189 




1 66a 

1 OO'* 


2ii 2t; 
Z*t 9 z ]p 


t79 ^ 

H/Z 


1ft 1 


OQIX 

zy 1 ^ 


51 39 


110 


296 

4u^O 


-014. 


2^ 26 
zp 9 zo 


' iiftl 
•f 0 1 


979 
z/z 


9nQA 
zuy*' 


39 8 


222 


464 


2ii9A' 

Z Z'* 


96 97 
ZO 9 z / 


iion * 


•♦Up 


UOp . 


8 25 


189 


260 


071 


27 2fl 
z / 9 zo 


(\ll \ 
0/ / \ 




1 9fl 

1 zo 


25 14 


209 


151 


144 


2ft 2Q* 
zo 9 z ^ 


«;ii6 
p*to 


lii9 
JMZ 


9nii A 


14, 16 


331 


361 


030 
\j J \j 


29 30 


597 


428 


1 69 
1 op 




187 




248a 


10 iT 


4qi 


45Q 


06 
oop 


54 19 


^?7 


\ 273 


016 


11 12 

J 1 9 JZ 


* *5Qft 


4ft 1 

MO 1 


1 1 9 
1 1 z 


19.30 


404 


/ 512 


1 08 
1 \J\J 


12 11 


505 
pup 


249 

ZMp 


zpov 

319*^ 


30,17 


. 28r-" 


550 


269* 


33,34 


639 


320 


17,50 


316 


584 


268* 


34,35 


462 


377 


085 


50,57 


395 


519 


124 


35,36 


416 , 


337 


079 


57,36 


286 ' 


409 


123 


36,37 


494 


237 


257* 


36,12 


148 . 


384 


236* 


37,38 


548 


319 


229 


12, 2 


180 


• 397 . 


217* 


38,39 


654' 


102 


252* 


2,23 ^ 


128 


375 


247* 


< 39;40 


430 


223 


207* 


23,46 


165 


412 ' 


• 246* 


40,41 


580 


319 


/261* . 


46,59 


249 




.15$ 


41,42 


581 


306 


.275* 


59.37 


313 


546 


23I* 


42,43 


512 


127 


385* 


37,26 


271 


374 


10/ 


43,44 


516 


213 


303* 


26,10 


243 


'577 


33?»* 


44,45 


499 


121 


378* 



31 
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TABt^^E 2--Contlnued 



•i9 





Form 3 
pairs 


Form 1 
N-i»80 


Form 3 


3-1 


Form 2 


Form 2 ' 


Form 3 


2-3 



721 
2i»^ 
250 
363 
309 

327 
320 
2i»9 
180 
322 
2i»9 
323 
'376 
266 
^31 
2'»3 



527 


351* 


45,46 


312, 


068 


46,47 




198* 


47.48 


i»6l 


098 


48,49 


klk 


165 


49,50 


568 




50.51 


^13 


093 


51,52 


^73 


224* 


52,53 


405 


225* 


53,54 


i»56 


134 , 


54,55 


i»06 


157 


55,56 


^♦38 


115 


56,57 


kkl 


,071 


57,5a 




177* 


58,59 


589 


158* 


59,60 


437 


194* 


60,61 



12k 


412 


312 


1, 1. r- 


302 


143 


643 


484 


159* 


612 


260 


352* 


589 


179 


410* 


516 


383 


133 


404 


225 


179 


507 


434 


073 


506 


214 . 


292* 


473 


190 


283* 


440 


217 


223* 


693 


, 527 


166* 


568 . 


395 


173* 


532 


320 


:ji2* 


5H 


451 


os^i 


517 


254 


263* 



Note.— Valges appearing In the table are correlations multiplied by 1000, 
* p •^.01 for two-tailed test. 



ERIC 



32 



TABLE 3 



30 



SUMMARY STATISTICS FQR TESTING EQUIVALENCE OF 
' VARIANCE-COVARIANCE MATRICIES BETWEEN 
PAIRS OF ITEM FORMS FOR VAI^IOUS 
SETS OF ITEMS 



\ 


Form 
compa r 1 son 


1 teres ^ 
fixed 


Model 


Reduction ^ 


df. h 1 P< 


df 1 X' 1 P< 



A. Item^Suiiset 1: 10-U» 16-l9*» 22-25 » ^0-i»3 



1 v« 2 


All 


1 CO 

• 53 


( 

1 Q 

20} .J 0 


. 0060 












1 CI 


393-00 


. poo J 










All 

A#fl T\tr ^ r tr A A \ 

Muj • prs* rroe" 




ZoU. 1 0 


. 000 1 












•DO 


• 0020 


0 
0 


80.50 


.0001 










♦ //39 


1 09 


170.27 


.0010 


2 vs ) 


A1 1 




395 .\09 


AAA 1 

• 0001 










F t rcl* HrrlAr 

voc^riQ uroer 


1 lift 




AAA 1 
• 0001 


13 


105.13 


.0001 




1 O 1 

131 


260.34 


. 0001 


9 


21.6i» 


^0110 




Third Order 


123 


223.90^ 


.0001 


8 




.0001 




* 

^B. Item Subset 


.2: i». 


20, 21, 


26-39 


« 




1 vs 2 

< 


Air 


153 


21i».27 


.0008 




19.22 






20» 21 » 26-39 


136 


195.05 


.0007 ' 


17' 


;3160 




27-39 


91 


151 -V 

57.?! + 


.0001 


us , 


, ^3.58 


.5^00 


1 v/s 3 


32-39 \ 


36 


.0131 


55 


9^.06 


.0010 


All 


153 


358.72 


.0001 






1 vs i| 


All 

Adj. prs. free' 


153 


237.85 


.0001 










150 


228.05 


.oqoK 


^3 


' 9.80 


.02i»9 




20» 21, 2*6-39 
27-392»3 


136 


I8i|.6l' 


-retJ3 5 




53.2i» 


.0001 • 




91 


111.85^ 


.0681 


59 


116.20 


.0001- 


2 ys 3 


32-39 * 


36 


^8.27+ 


.0831 


55 


63.58 


.2000 


AH 


153 


337.30 


.0001 










First order free 


139 


^^8.57 


.0001 


\k 


88.73 


.0001 




Second order free 


127 


2i»0.77 


.0001 


12 


7.80 


.8000 




Third order free 


116 


230.67 


.0001 

• 


11 


10.10 


*5210 



0 
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TABLE 3— Continued 



Form 
comparison 


Items 




Model 


Reduct Ion 




fixed 




df 1 1 p< 


1 df 1 v2 


1 P< 




C. 


\ 

1 tem 


Subset 3^ ^^-61 







I 


vs 


2 


AH- 




171 


231.87 


.0013 






e 


1 


vs 


3 ^ 


All 




171 


501.79 


.0001 








1 


vs 


k 


An2 




171 


201.96+ 


.0528 








2 


vs 


3 


All 




171 


471.36 


.0001 










/ 




First order 


free 


154 


294.79 


.0001 


17 


176.58 


.0001 






Second order 


free 


138 


239.56 


.0001 


16 


55.32 


.0001 








Third order- 


free 


123 


207.50 


.0001 


15 


32.00 


.0070 



0. Item Subset 4: I-3, 5-9 



1 


vs 


2 


-1 

All 




36 


42.48+ 


.2119 








1 


vs 


3 


All 




36 


.74.81 


.0002 








1 


vs 


4 


All2 




36 


39.43+ 


.3193 








2 


vs 


3 


All 




36 


82. KO 


.0001 














First order 


free 


30 


55.63 


.0030 


6 




.0001 








Second order 


free 


25 


44.84 


.0087 


5 


10.79 


.0560 






f 


Third order 


free 


20 


33.10+ 


.0329 


5 

• 


11.73 


.0410 



Note.— The superscripts denote the following: (1) Items have been rese- 
quenced for one of the two forms, (2) Items remain grouped Identically and In 
the same sequence for the two forms, (3) Items have been resequertced for the 
two forms, and (+) model provides for a sufficient fit of the data. 



Form 1 



Items 



TABLE 4 

ITEM SET ONE INTERCORRELATIONS* FOR FORM ONE AND FOUR 



Form k 




I terns 



10 
11 
12 
13 
\k 
16 
17 
18 

19 
22 

23 

2k. 

25 
i|0 
ill 
i|2 
i»3 



10 11 12 13 '\k 16 17- 18 19 22 23 2k 25 



59 

50 

36 

26 

33 

37 

27 

29 

19 

11 

28 

23 
23 
22 
16 
18 



56 53 
59 

63 

k5 53 
33 3k 
36 
3k 
33 
20 
21 
18 
28 
22 
21 
13 
18 

13 



k2 
i»6 
52 



38 

39 
36 

23 
17 
]k 
32 
19 
18 
16 
15 
20 



17 
25 
17 

V 32 
k\ 

i»2A 33I 
^k 32^ 
35 32 
35 25 
26 31 

19 20 
26 ifO 
23 21 
22 26 

20 23 
20 20 
22 25 



28 39 

36 '38 

38 3k 

ijSA 38 

i,3l 33B 

57 

6k 

kO. 38 

21 28^ 

3k 31 

26 16^ 
k2 37^ 

30 23 

27 2k 
20 19 

28 22 

31 29 



37 
38 

29 

ko 

31 
k7 
56 

502 

28 

15, 
383 

Z7 < 
23 

17 » 
27 



3k 2k 
36 29 
23 ^2 
29 19 
28 28 

17 23 
I2C 17 

k\^ 



10 39 
25 35 



17 
20 



^♦0 
i»8 



332 

13^ 36 
37 
33 



kO^ 
kS^ 
33 29 
30 31 
27 28 
32 35 



23 ^k\ 

20 i»5 

19 ^♦5'' 

18 5k^ 
25E 27 
kk 20F 

39& i,i»7 

26" 31 

19 36 
25 31, 
37 ki 



25 


ko 


k\ 


k2 


it) 


20 


26 


22 


21 


27 


32 


18 


19 


31 


2i» 


28 


17 


18 


22 


21 


26 > 


20 


17 




2i» 


18 


25 


26 


27 


26 


27 


35 


31 


35' 


2i» 


22 


2k 


20 


25 


31 


28 ' 


20 


2k 


22 


33 


5^ 


31 


20 


22 


30 




27 


ko ' 


19 


2i» 


Sk^ 


27H 


25 


20 


32 


237 


26 


29 
37' 


30 


36 


328 
27' 


398 


22 


33 




53 


k7 


37 


50 




kS 


35 


31 


ko 


k5 




kk 


35 


k7 


.38 


kz 


\ 



Nou.— The numerical super^rlpts Indicate the corresponding Intercorrelatlons for- 
Items that were adjacent In only one of the two forms. Form 1 parls are (18.18). (23.2i»). 
• •"<'^(?^»25). Form k pairs are (IM.6), (18. 2i»), ^(19,22), (23,25), and (25.40). The 
aiphabetlc superscripts denote those Item pairs t;hat appear as second order Item palrs^ 
(separated by one other Item) In only one of the two forms. Fori^ 1 second order Item pain 
•re (17,19) and (22,24). Form k second order Item pairs are (13,16), (14,17 . (17.24). 
,(19,23), (22,25 . 23.40 . and (25,41).- V " 
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•Tabled values are Intercorrelatlons multiplied by lOO. 



TABLE 5 



FORM ONE AND FORM FOUR INTER- ITEM CORfliLATIONS 
FOR ADJACENT ITEM PAIRS WHERE ONE 
ITEM DOES NOT BELONG TO THE 
ITEM SUBSET 



Form 1 


Correlation 




Form k 


1 ^ 

CorreTatlon 


pairs 


Form 1 


Form k 




pa 1 rs"^ 


Form 1 


1 Form A 




Item 


Subset 1: 10-1A» 


16-19, 22-25, A0-A3 




9.10 

15J6 
J9,20 
21,22 
25,26 
39, AO 


21 

35 
51 

A5 
A3 
A3 
27 
39 


16 , 

26 

27 

.. 38 
37 
25 

' . 21 
36 




3,10 , 
19,31 
A3, 32 


22 
A3 
38 


36 
35 




lUm 


Subset 2: A^ 


20, 


21, 26-39 






•3, A 
A, 5 


32 
A5 


28 
AO 




A,2A 
39, AA 


30 
27 


33 * 
26 



Note.— Values appearing In the table are correlations multiplied by 100, 



36 
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TABLE 6 



Form 



ITEM SET TWO INTERCORRELATIONS? 
FOR FORMS ONE AND TOUR 




Form k 



I terns 



20 
21 
26 
27 

_U«mt 28 

29 
.30 
31 
32 
33 
3^1 
35 
36 ^ 

37 
38 . 
39 



1 



56' 

ik^ 61 
27 33^ 



B 



31 
28 

25 



45 

3^ 
38 



kO^ 33 
5V 31, 

38^ 
1,9c 

47 34 



29 42 
26 45 
29 26 



24 
27 
16 

33 
26 
24 
20 



34 , 

28 

27 

27 

34 

25 
27 



48 
45 
52 
34 
34 
36 
32 
30 
39 
34 
32 



35 
32 
34 
23 
27 
25 
29 
.32 
27 
22 

23, 



27 


28 


2? 


30 


?1 


?2 


33 


,3^ 






?7 


38 


3? 


49 


47 


20 


19 


30 


24 


37 


27 


14 


28 


4 

21 


11 


11 




38 


27 


25 • 


28 


33 


38 


"30 


23 


39 


37 


28 


19 


53C 


49 


38 


48 


51 


31 


33 


39 


36 


37 


56 


43 


41 


46 


37 


24 


30 


32 


18 


15 


18 


22 


24 . 


25 


22 


14 


6^ 


71 


41 


45 


46 


21 


27 


'33 


27 


31 


37 


28 


30 




47 


^5 


46 


30 


29 


32 


34 


37 


35 


30 


36. 


51 


50 




53 


51 


28 


27 


30 


34 


28 


37 


35 


39 


42 


45 


60 




61 


27D 


23 


27 


31 


23 




42 


37 


51 


52 


63 






3l3 


33^, 


37 


33 


34 


48 


^8 
32 


34 


33 


32 


40 


.38^ 


58 


51 




36 


42 


39 


34 


36 


38 


37 


37 


44^- 




61 


36 


32' 


36 


33 


37 


41 


44 


37 


31 


39 




.65 


56 


. 38 


'49 


37 


44 


32 


37 


38 


33 


30 


?6 


42 


50 




42 


47 


40 


33 ' 


34' 


34 


25. 


25 


27 


29 


^34 . 


40 


35 


> • 


50 


33 


24 


35 


35 


41 


4^1 


43 


34 


^2 


44 


42 


37 




53 


43 


33 


34 


34 


3^ 


31' 


38 


40 


37 


31 


33 


&2 




49 


39* 


39 


40 


34 


38-. 


37 ' 


38 . 


^1 


41 


26 


49 . 


49 * 



Not«."Tho numer-lcfll supericr^ts Indicate the corresponding Intercorrelatlons for Items that 
w«re adjacent In only one of two forms. Fo'rrtj 4 paJrs are (4,^0) and (21,26). The only adjacent 
pair for Form Kls (31',321.' The alphabetic superscripts der\ote those Item pairs that appear 
•S lecond order Item pairs (separated by/^ne other J tern) In only one t)f thi two formsT-^orm 4 se- 
tond orde^ Item pairs are (4.21), (20,26) arid (21,27), Form I second order Item pairs arc (30,32). 
(3U33) . - * . ' . ' ^ . 



^Tabled v»lu«s are correlatl(?ns multiplied by loq. 
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TABLE 7 

RESIDUALS DERIVED IM riTTING SIX FACTOR 
V MODEL BASED ON FORM 1 DATA 
FOR THIRTY -FOUR ITEM SET 







« 

\ 


Residual 


< • 


{ 


Item 












pa ir 














Teacher 




^Administrator 






• 


Form 1 


Form k 


Form 3 


> « 


A 

A« 




IteiP'palrs 


for teachers form. Form* 1 


and Form *♦ 


in If 


f\UL 

-0*»o 


-033 


-018 


-026 




11 12 


-00/ 


01 *♦ 


-006 


-014 




1? W 




057 


052 


143 










-026 


-050 




16.17 


-006 


-058 


-081 . 


128 




17,18 


-177 


01 1 


-183 


-013 




20,21 


-109 


-125 


-119 


-106 




22,23 


-166 


-199 


-212 


-121 


■ .:+ 


26.27 


-Oi»3 


-013 


-032 


-043 




27.28 


-003 


-Oil 


-029 , 


015 




28,29 


-028 


003^ 


014 


120 


• 


29.30 


-049 


-045 


-062 


004 




30.31 


loa^i 


•001. 


-095' 


' . 006 




32.33 


113 • 


031 


073 


205 




33. 3 A 


024 


044 


-030 


114 




3^.35 


-095 


-058 


-150 


-047 


4- 


35.36 


-201 


-031 


-142 


-088 




36.37 


-129 


-059 


-214 


048 




37.38 


-186 


-201 


-208 


• ^^022 


+ 


38.39 


-208 


-184 


-191 


-131 




AO. ill 


-154 


-117 


-J75 


-037 




k\M 


-115 . 


-lOA 


-*70, 


-050 


+ 


A2.1i3 


-133 


-«06I 


-102 


n 148 
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TABLE 7--Cont Inued 







1 tern , 
P«lr ♦ 


• 

Residual 




Teacher 




Admfn 1 strator 








Form I 1 


Form A j 


Form* 3 


B. 


Item pat rs 


For Form ' but 


not for form 1 


or Teacher form 



4,20 


013 


-030 


>289 




' 097 




010 


-024 


-124 


+ 


-046 


I8,2i» 


041 


022 


s -158 


+ 


126 


2i., J 


029 


022 


-014 


+ 


071 


21,26 . 


-001 


017 


-077 


+ 


076 


31 J9 


-040 


000 


-010 




-019 


19.22 


-030 


-008 


-173 


+ 


-055 


23.25 


-208 


-132 


-384 


+ 


-200 


25.40 

m 


024 


035 


# -039 




-012 


\ 

C. Item pairs for 


Teacher 


form and Form 1 , 


but 


> ... 
not Form 4 



18,19 


-127 


-182 


-047 




-153 




19.20 
<. 21,22 




-141 
-064 


-100 
-025 


+ 
+ 


-127 
008 


+ 


23.24 


-092 


-063 


146 


+ 


047 


+ 


^4.25 


067 


-060 


129 




156 




25.26 


-106 


-085 " 


086 




-003 


+ 


31.32*» 


079 


106- 


150 




-052 




39.40 


-081 


-061 


-023 




-035 





3y 



TABLE 7— ContNui^d 



37 



I tem 



pair 














Teacher 






« 






Form 1 




Korm 5 






0. ^ Item pairs for Forftl 3, 


Dut not Form i> Form 


or Tc«ch«r 


form 


10,26 


-062 


-013 


Wf 


-282 




- 


11,21 


031 


009 




i| 053 






11. ^»2 


00^» 


013 


-0o2 


046 






12.36 


' -085 


-OU 


mi 
01 1 


-119 ^ 


+ 




13.^0 


027 


025- 




003 


+ 




1^.25 


-006 


-02i» 


001 


-154 


+ 




14.16 


-010 


-024 


-027 


-046 


+ 


''■4 


17,30 


-036 


021 


-062 


, -166_ 






18.21 


028 




008 


• -100 


+ 




19.30 


0)9 


--0I2 " 


"\ 074 


-218 


+ 




20.32 


057 


051 


-Oil 


-035 


+ 




26.37 


027 


. -041 


'^052 


-131 


+ 




27.31 


038 


022 


062 


-1 j2 


+ 




27.38 


001 


-013 


-004 


002 « 






29.35 


-038 


-096 


-t)62 


-161 . 


+ 




35.^3 


-0i»6 


-050 t 


-147 


-206- 


• + 




^♦0,'»2 


-026 


-016 


-Q07 


-039 


+ 





Not*.— The values appearing In the table are 1000 timesi, a« large as 
the actual residuals. Yhe (+) and (-) symbols are used to denote agreement 
and disagreement with anticipated patterns assuming Item sequence Inflates- 
thffcpvarlanCifcT'for adjacent Items. , 

\ \ 

•COFAmM computes the residual as I - § where £ Is the estimated covar4ance 
matrix and S the safnole covarlance matrix.' Thus. IH part A. Form 3 residuals 
should be less negatTve than residuals for Forms .I.tfnd 4 and for the teacher v 

covarlances as anticipated. Form 4 residuals 
should., In the presence of the anticipated effect, be mora negative for the ' 
restduats reported In part B. In part C, residuals for Forms 3 and 4 were 
.•xpacted to be less negative than the other residuals.' 

^These Itertii appeared as a4Jacent Items on Form 3. 
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TABLE 9 

FORM ElVE MAJOR 
* . FACTOR LOADINGS 



U«nrC 



Group tng 



Form 1 



Form 5 



3 ^ i 



Factors 



T 



Tn — w 



VT 



T 



To 

12 
1 



1 

16 

17 
18 

21 



2i» 



22 
23 

25 



26 



27 
28 

39 
30 

31 - 



To- 
ill 

A2 



32 
33 
3A 
35 
36 
37 
38 



5^ 

ii8 
k\ 

57 
61 
66 
38 
59 
63 
AO 

53 



AO 
50 

69 
36 



>0 
55 
5A 
58 



23 
AO 
55 
59 
60 

65 



58 

69 
69 
51 
56 

A7 
A9 
53 



1^ 
58 



50 
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TAitC 

QUnTIOMHAlKI III 



• f 

l« IfhAvInf l» • mmnnmr whUh It contlttant with thm luqml rights anrf ratponti* 

bll Ittat ortMCh«rt. 
I. Orf«nltlnf th« «UttnKH« •nvlroomtnt to •Mphatlia f>mr4t rathar than 

purtltN^ntt« 

1» Coordinating Ind I v Idual . and group Instruction and tupplwnanUry actlvltlat 
within tha conflnat of tchadul Ing constraints. 

N«|ntaln|ng clattrooM racordt to facllltata clattroon managamant. 
$• CtCabllthIng procidurat to maintain an ordarly olatt. 
i« iMplmntIng ttrataglat to ancouraga potltlva ttudant attltudat to««rd 

iMrnIng, 

f. OatarMlnIng ^han and how to apply group matNidt of ditttlpllna. 
!• Analyting tha problam baKavlort of ttudantt. 

9« Upl tenanting procadurat to ratolva Individual bahavlor prxiblamt, 
I0« Salacling Instruct loha I objxtlvat which aca ralavant to foog-ranga goals. 
M. Stating Instruct lynal object Ivat In tarmt of obtarvabia ttudant bahavtort. 
It. Cttal^l Uhlng aspactad ttudant parforranca lavalt for Inttruct lonal objactlvat. 
I|« using cognltlva (a.g., ability and achlava<i««nt) and non-<«gnltlva (a.g.« 

attltudl(\al, v^lua. and toclologlcal ) Information about ttudantt taattabltth 

priorltlat for g^alt and objactlvat. 
1%. Idantlfylng Individual and group ralnforcart Nf>lch can b« utad to notlvata 

ttudantt. 

1$. ^acognlilng ttudantt with tpaclal cognltlva (a.g.« laarnlng dl tabl 1 1 1 lat) 
or affactlva (a.g., amotlonal) problamt who naad th« attantlon of tchool 
and comrainlty tpaclal Ittt. 

Oatarmlnlng wf^athar tha laarning catk Idantlflad In an objactlva raqulrat 
prfmardy m«««>r I tat Ion, formation of a naw concapt. problam solving, tynthatit 
ate. 

II. Idantlfyino tha orar^qultlta tklll apd hnowladga nacattary for a ttudant to 
I achlava o6J«<tlva . ^ 
II* Iraalilng down • laarning tatk Into^ camponantt In ordar to acco«miodata ttudantt 

with diffarant ^blllt and Mllltlat^ 
19. Oatlgning Inttruct lona I programt to («»aat tha Individual naadt of ttudantt. 

10. Arranging Inttruct Ion to that ttudantt can mpva In an ordarly ff^nnar toward 
/ tha achlavamant of an objactlva. 

II* Applying laarnlfig princlplat whan planning and guiding laarning'' act IVItlat. 
!!• Incorporating rocani lubjaci Mat tar davalopmantt Into tho Inttfuctlonal 
pnocatt. 

1|. tJiparl»antlng with diffarant Mthodt of taachlng. ^ 

1^ Applying knowladgarof Child and adolafcant davalbpinanC whan planning and 

guiding laarning actUltlat. 
IJ. Uting nawly dtvalopad curriculum procadurat and Mtarlalt. 

11. evaluating tha appropr latanaM of Inttruct lonal Mtarlalt (a.g.« tantbookt). 
Vim Contacting Information on a ragular bat It cone arn Ing ttudant progratt toward 

tha achlava^nt of (nttruct lonal objactlvat. 4 ; " 

II. Providing fatdback to ttudantl on a ragular hat It concarning thair progratt 

t^ard ach laving Inttructlonal objactlvat* 
19* ^Utlng ptrformanca l^for^tlon to pinpoint tha natura of loarning diff Itultlgg* 
)0« Hattructurlng Inttructlon whan irataachlng It nocattary* 



Numbar 



ItM daicrlf tloA 



|l« UtIng Infofw^tlon about ttudant parfomvnca to Ottlin tubta^ant loarninf 

actlvltlat (a*t-t t-aaiadlal and taquantlal)/ 
32. Intaractlng with ttudantt from • diffarant aoclol and othnlc Uclifround 

than thalrt. V 
3)« Employing group actlvltlat to >romota widatpraad ttudant participation. 

Intaracting with ttudantt In %«ayt to promota mutual undarttanding and truat. 
3$. UtIng tachnlquat of quattlonlng to ancouraga highar lavalt of ttudant 

thinking (a.g.« analytit* aynthatit* and avaluatlva fhlnklng), 
3(. fretanting claar and logical varbal Inttructlon (a.g.* lacturat). 
37« JImployIng procadurat to ancouraga tal f -d Iractad ttudant^ bahavlort. 
)l« fincorporat Ing' valua clarification actlvltlat Into tha Inttructlonal procatt. 
35« luting tachnlquat' to ancouraga potltlva ttudant tal f-concaptt and faal Ing t 
Ifif worth. 

^0. XlUIng It'arnlng cantart, audlo*vlti»al aldt« talavltlon« garnat* and othar 

multlmadia mathodt to prcnuta ttudant laarning, 
^1. IncorRorat Ing comniuni ty fac 1 1 1 1 lat and ratourca paopla Into tha Inttructlonal 

procatt. * / 

^2. Ut^ng para-profattlonalt* tutort* and othd| Inttructlonal paraon^al In 

Conducting clattroom ralatad actlvltlat. ' 
^3« Davlting Inttructlonal matarlalt and actlvltlat. Including ramodlal. 
kk* Salacting tattt which avtch ttatad ojbact Ivat. 

^S. Conttructing objactlva cognltlva mt Itamt (a.g*. ami t lpla<*Cholca and 

matching). 

Conttructing non«obJmct Iva cognltlva avaluatlon procadurat (a.g. « ataay 
Itcfiita parformanca aatattmantt* and product avaluatlon)* 
^1. Intarprating convnarclal achlavamant tatt acor'at. 

^B. Calculating normatlva tcorat tuch aa ataantt madlant* parcantllat* ttanlnat* 
ate. % 

Conttructing noni:ognltlva attattmant davlcat (a.g.i attltudat and valwat). 
50. Improving pratant attattmant davlcat and procadurat. 

$1. Intarjratlng tha-ratuitt of tatting and avaluatlon to ttudantt* thalr pargntt 

and to othar prof att lonal t In tamt aach can undprttand. 
51* UtIng ttudant faadback (a.g»i opinlont) to avaluata and ravlta Inatructlon* 
S3* UtIng ttudant achlavamant data collactad throughout tha Inttructlonal 

prpcatt to avaluata and ravlta Inttructlonal matarlalt and procadurat. 
%\m Oatannlnlng whathar objactlval hava baan Obtalnad for which porformanc^ 

ttandardt cannot ba tat (a.g.i ren-cogn|tlva objactlvat). 
SS« 0avltlng acCurata« objactlva mathodt for parlodlCally tummarliing and 

raporting ttudant achlavamant (a*g.« gradat). 
Sft* Implamanting ^^cadurat to aniitt parontal tupport of your Inttructlonal 

program. * 

$7* Exchanging Information about ttudantt with parantt on • ragulor batit 

(••f«« progratt dif f lcultlat« tpaclal problamt). 
$1* Intaracting with parantt from 0 diffarant goclal and othnic ^ckground 

than thalrt. 

59* PItcuttIng thalr Inttructlonal tuccattat. naadt pnd rroklamt with achomi 

admlnlttrator^t In a non*|hraatanlng "^nvlronmant* 
iO. Communicating thalr goalt and objact I vat to tha pukJIcC \: 
* 41* Pacalving partlnant Information about tha actUltlat of govarnmant« tchool 
*dmlnlt|ratlon« community, and profaatlonal Ofganltatlon (l) 

' : \ 1.1 
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